Method for enhancing vehicle camera image quality

ABSTRACT

A method of image enhancement for a vehicle vision system includes providing a camera at the vehicle and providing a processor operable to process image data. Multiple frames of image data are captured with the camera, and contrast is enhanced in image data by tone mapping. As the vehicle moves, contrast thresholds are tracked within the captured frames of image data with respect to image flow caused by the vehicle&#39;s movement. Image data of a first frame of captured image data may be passed through two individual image transfer functions to generate a first transferred frame of image data. The first transferred frame may be blended with a second frame of image data. Presence of an object is detected in the field of view of the camera, and an output is generated responsive to detection of the object present in the field of view of the camera.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/619,630, filed Jun. 12, 2017, now U.S. Pat. No. 9,774,790,which is a continuation of U.S. patent application Ser. No. 14/343,937,filed Mar. 10, 2014, now U.S. Pat. No. 9,681,062, which is a 371national phase filing of PCT Application No. PCT/US2012/057007, filedSep. 25, 2012, which claims the filing benefit of U.S. provisionalapplication Ser. No. 61/539,049, filed Sep. 26, 2012, which is herebyincorporated herein by reference in its entirety.

FIELD OF THE INVENTION

The present invention relates to imaging systems or vision systems forvehicles.

BACKGROUND OF THE INVENTION

Use of imaging sensors in vehicle imaging systems is common and known.Examples of such known systems are described in U.S. Pat. Nos.5,877,897; 5,796,094; 5,670,935 and/or 5,550,677, which are herebyincorporated herein by reference in their entireties.

SUMMARY OF THE INVENTION

The present invention provides a vision system or imaging system for avehicle that utilizes one or more cameras to capture images exterior ofthe vehicle, such as forwardly or rearwardly of the vehicle, andprovides for enhanced image processing to detect objects in poorvisibility conditions, such as in dense fog or the like.

The vision system may enhance the image processing by amplifying thecontrast in the captured images by brightness transfer functionfiltering and exposure stacking and tracking contrast thresholds orfeatures within the captured images, such as on a frame-by-frame basisas the vehicle travels along a road.

These and other objects, advantages, purposes and features of thepresent invention will become apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a plan view of a vehicle with a vision system and forwardfacing imaging sensor or camera that provides a forward exterior fieldof view in accordance with the present invention;

FIG. 2 shows images captured by the forward facing camera and processedby the vision system of the present invention;

FIGS. 3(a) and 3(b) show graphs showing histograms of luminancedistribution for the vision system, with FIG. 3(b) showing the histogramof the original image's graph of FIG. 3(a) as spread into the highestpossible dynamic range of the target system which equates to a contrastamplification of the present invention;

FIGS. 4(a) and 4(b) show graphs of brightness transfer functions A (FIG.4(a)) and B (FIG. 4(b)) which find use in the flow chart of FIGS. 5 and6, with the brightness transfer function A enhancing the brighter areasand dampening the darker ones, and with the brightness transfer functionB decreasing the medium illuminated areas, and with the upper end atless of 100, whereby the overall illumination becomes decreased by thistransfer function;

FIG. 5 shows a flow chart of the image enhancing and processing stepsaccording the invention, and by mapping/stacking an illumination reducedimage scene on top of a contrast enhanced image the dynamic range of theimage increases: overexposed areas appear less bright and underexposedmore bright, which leads to acknowledge details in the scene easier;

FIG. 6 shows a flow chart of the image enhancing and processing stepsaccording the invention as to be used in a vehicle vision system,supporting machine and human vision driver assistant algorithms; and

FIG. 7 shows an example of how images may be altered when processedaccording to the flow chart of FIG. 5, whereby it becomes apparent thatthe process turns out more contrasts of possible objects on foggyweather conditions when comparing Image(t₀) and Image_(h)(t₀).

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to the drawings and the illustrative embodiments depictedtherein, a vehicle 10 includes an imaging system or vision system 12that includes at least one imaging sensor or camera 14 (such as forwardfacing camera at the front (or at the windshield) of the vehicle), whichcaptures images exterior of and forwardly of the vehicle (FIG. 1). Theimaging system 12 is operable to process (such as via an imageprocessor) image data captured by the camera 14 to present these on adisplay or to detect objects and/or headlights of approaching vehiclesand/or taillights of leading vehicles in the field of view of the camera(such as for use in an object detection system of the vehicle orcollision avoidance system of the vehicle or headlamp control system ofthe vehicle or adaptive cruise control system of the vehicle or lanechange departure warning system of the vehicle or traffic signrecognition system of the vehicle or driver assistance system of thevehicle the like).

The image processor of the vision system 12 is operable to processcaptured image data, such as to detect and identify objects forward (andoptionally sideward and/or rearward) of the vehicle during normaloperation of the vehicle. In poor visibility conditions, such as foggyconditions and/or heavy snow fall conditions or the like, objects may bedifficult for the driver to see and may be difficult even for the imageprocessor to detect, even when image processing algorithms for lenspollution detection (such as similar to that described in U.S.provisional application Ser. No. 61/616,126, filed Mar. 27, 2012, whichis hereby incorporated herein by reference in its entirety) come intouse. For example, and with reference to image “A” in FIG. 2, during lowvisibility conditions, such as fog conditions as shown, it is difficultfor the driver of the vehicle to detect the person and dog at the sideof the road ahead of the vehicle and beyond the principal illuminationarea of the vehicle headlamps (set at low beams for seeing in the fog).The image processor may process the image to detect objects, but, andwith reference to image “B” in FIG. 2, normal image processing may notdetect the object of interest (the person and dog in this example) dueto the poor visibility conditions. Typically, the object detection maynot work feasibly when the image contrast falls under a certain level.In order to increase the detectability of such objects in poorvisibility conditions, the vision system of the present invention isoperable to enhance or increase the contrast of the captured images sothat any objects in the field of view of the camera are darkened toenhance the detectability of the objects by the image processor or toease the visibility of objects to the driver of the vehicle.

As can be seen with reference to images “C” through “F” in FIG. 2, asthe contrast is increased, the side markers or posts along the side ofthe road and the object of interest (the person and dog in this example)become darker and, in this example, the object moves relative to otherfixed objects in the captured images (see images B-F in FIG. 2 and notethat the person and dog approach the fixed road marker in the capturedimages), and thus the image processor can detect the presence of thefixed and moving objects and determine if they are objects of interestto the driver of the vehicle and generate the appropriate signalresponsive to such detection and determination or identification. Forexample, the system, responsive to such an object detection, maygenerate an alert to the driver or may adjust the headlamps accordinglyor may display the detected object on a display screen for viewing bythe driver (particularly for backup assist systems where the object isdetected rearward of the vehicle during a reversing maneuver). Thus, byincreasing the contrast in captured images, the vision system canenhance detection of objects in the camera's field of view that mayotherwise go undetected. The system may be operable to increase thecontrast in the captured images responsive to a user input or to adetection or determination of a low visibility condition, such asresponsive to a signal from a rain sensor or the like that is indicativeof detection of a foggy condition or such as responsive to imageprocessing of the captured images to determine that the vehicle is infoggy driving conditions (such as by utilizing aspects of the visionsystems described in U.S. Pat. Nos. 4,973,844; 5,796,094; 5,877,897 and6,353,392, which are hereby incorporated herein by reference in theirentireties).

It is known to provide image contrast enhancing for photographs (such asphotographs taken by hand held digital cameras or astronomicaltelescopes or the like), and such enhancements may be done by knowncomputer based tools for editing images. Today, nearly every operatingsystem, library, presenting program and/or the like provides at leastbasic image editing functions. Professional photo editing programs likeCoralDRAW®, Gimp® or Adobe Photoshop® provide a wide range of imageediting and enhancing features. Typically used for contrast enhancing isthe editing of the contrast histogram. This can be used to exposeobjects stronger. A function used especially to do this is “ContrastEnhancement through Localized Histogram Equalization” (seeCromwell-intl.com: http://www.cromwell-intl.com/3d/histogram/, which ishereby incorporated herein by reference in its entirety). Even nightimages can become contrast enhanced in a way that low illuminatedobjects turn out more visible. Such algorithms used in consumer computerprograms for image enhancing are typically used in individual pictures,and are not meant to be used in real time applications.

Image quality improvement in poor visibility conditions is known fromairborne weather surveillance pictures for reworking pictures taken incloudy (foggy) situations. The best results were achieved by Oakley etal. when contrast enhancement algorithm in conjunction with a temporalfilters came into use (see Image Processing, IEEE; “Improving ImageQuality in Poor Visibility Conditions Using a Physical Model forContrast Degradation,”http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=660994, byOakley, J. P. and Satherley, B. L., February 1998, which is herebyincorporated herein by reference in its entirety). The base was aphysical model on fog reflection.

Attempts have been made to do video contrast enhancements such as in“Contrast Enhancement Using Brightness Preserving Bi-HistogramEqualization” by Yeong-Taeg Kim (Consumer Electronics: IEEE: “ContrastEnhancement Using Brightness Preserving Bi-Histogram Equalization,” byYeong-Taeg Kim, February 1997, which is hereby incorporated herein byreference in its entirety). This requires real time processing. Demandfor this was and is in applications for the likes of television images,images providing medical devices, military engineering and/or the like,and Kim et al. suggested “Partially Overlapped Sub-Block HistogramEqualization” to be used in cameras (Circuits and Systems for VideoTechnology, IEEE: “Partially Overlapped Sub-Block HistogramEqualization”http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=915354, byJoung-Youn Kim, Lee-Sup Kim and Seung-Ho Hwang, April 2001, which ishereby incorporated herein by reference in its entirety). Also, Marsi etal. were able to simplify algorithms by attempting recursive rationalfilters (Imaging Systems and Techniques, 2004; IEEE InternationalWorkshop: “Real Time Video Contrast Enhancement by Using RecursiveRational Filter,”http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=1397276, byMarsi, S., Ramponi, G. and Carrato, S., May 14, 2004, which is herebyincorporated herein by reference in its entirety), and Wang et al.suggested the use of weighted thresholded histogram equalization forfast processing (Consumer Electronics, IEEE: “Real Time Video ContrastEnhancement by using Weighted Thresholded Histogram Equalization”http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=4266969, by QingWang and Ward, R. K., May 2007, which is hereby incorporated herein byreference in its entirety). Another challenge is the noise, a commonproblem on electronic cameras; Starck et al. published a procedure to donoise reduction by curvelet transforms in 2003 (Image Processing, IEEE:“Gray and Color Image Contrast Enhancement by the Curvelet Transform,”http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=1208320, byStarck, J.-L., Murtagh, F., Candes, E. J. and Donoho, D. L., June 2003,which is hereby incorporated herein by reference in its entirety).

It is also known to use infrared systems or low light amplifying systemsin vehicles. Earlier systems have used infrared cameras alone, and somesystems additionally use infrared headlights to light up the area infront of the vehicle (invisible for the human eye) which makes that areaeasier to detect with the infrared camera. Infrared cameras may provideenhanced performance in object detection in dense fog conditions due toits physical principal and the detected wave length have the intrinsicproperty to interfuse fog, so objects in fog can be detected and/orvisualized.

State of the art automotive driver assistance systems typically providethe driver with useful information of the vehicle's environment,including the traffic or objects in front of, to the side of andrearward of the vehicle. Typically, there are additional warnings orimage overlays for highlighting hazards, especially those in the drivingdirection of the vehicle and in the anticipated path of travel of thevehicle. Obstacles or pedestrians that are in the way or path of thevehicle or tend to step into the path of the vehicle may be highlighted.Systems which also do active interventions such as braking or collisionavoidance maneuvers are also known. For distinguishing pedestrians fromother objects and for predetermining their walking direction and speed,the detected objects need to be tracked over a certain time. Also,analyzing shapes or markers of walking or standing pedestrians is knownin the field of automotive vision systems and image processing. Due tothe vehicle's own movement, the objects in the captured images flow ormove over successively captured images (optical flow). For example,external or outside objects (even stationary objects) move through theimages taken from a front facing vehicle camera as the vehicle travelsalong the road. Algorithms for tracking objects under driving conditionsare also known. When a vehicle drives through a turn, the optical flowalso behaves in a turned manner. That turn can be anticipated by theknowledge of the steering wheel's angle and a kinematic model of thevehicle's curve behavior. The optical flow speed directly translatesfrom the vehicle's ground speed given by the odometer. Alternativelyknown algorithms may determine the optical flow direct from the imageflow without the previous mentioned input from the vehicle.

For enabling the above mentioned pedestrian and obstacle acknowledgingand tracking algorithm to work properly, especially to be able tohighlight a hazard or warn the driver or intervene (such as via brakingor cruise control adjustment or the like), it is necessary to receivesufficient images. In foggy driving conditions or during heavy snow falldriving conditions, cameras in the visible spectrum deliver images ofinsufficient quality. The present invention provides enhanced imagequality of visible spectrum cameras, especially the dynamic range of theresulting image, so that the driver assist system algorithms can workproperly and/or display the processed image to the driver as animprovement to his or her view in such limited visibility conditions.This is achieved without the need of additional cameras using differentlight spectrums (such as infrared sensitive cameras or the like) orother sensors for the same purpose or high dynamic range (HDR) cameras.

The present invention thus provides enhanced image quality in poorvisibility conditions captured by a non HDR camera by amplifying thecontrast details in the captured images by generating a pseudo HDR imageout of current and historical image components by tone mapping. Thesystem then tracks the contrast thresholds/features within the capturedimages with respect to the image flow caused by the vehicle's movement.This process is repeated on a frame-by-frame basis to detect andidentify objects in the camera's forward field of view, as can be seenin FIGS. 5 and 6. At every loop the historically (previously enhanced)image (Image_(h)(t₀-n))) passes two individual image transfer functionsand then becomes superpositioned (or mapped, merged, blended or stacked)by the currently captured frame (Image t₀)). This tone mapping method iscalled image stacking, exposure fusion or exposure blending. The mappingratio of how much of the historical image (Image_(h)(t₀-n))) becomeskept and how much of the current image (Image t₀)) becomes mapped in isfreely selectable between 0 and 1. In the example in FIG. 5, 20%/80% waschosen for a data frame rate of 30 frames per second. Slower frame ratesmight require a shift into a stronger influence of (Image t₀)). The usedimage enhancements shall not be limited to these shown in the example ofFIGS. 5 and 6.

The brightness transfer function A (FIG. 4(a)) enhances the brighterareas and dampens the darker ones. This equates to a histogram spread(Dynamic Range Increase), such as shown in FIGS. 3(a) and 3(b), of thehistorically image (Image_(h)(t₀-n))). The brightness transfer functionB (FIG. 4(b)) decreases the medium illuminated areas of the currentlycaptured image (Image t₀)). The upper end is at less than 100%. Theoverall illumination becomes decreased by this transfer function. FIG. 7shows that already after this step the discrimination between the object(person with dog) and surrounding (foggy) area is improved. Bymapping/stacking an illumination reduced image scene (currently capturedimages) on top of a contrast enhanced image (historical image), thedynamic range of the image increases as to be seen in FIGS. 3(a) and3(b). Overexposed areas appear less bright and underexposed more brightwhich leads to acknowledge details in the scene easier (see FIG. 7).After consecutive loops it may come to a blooming effect or halo at theborderline of areas with high contrast. This effect may be enhanced bysome blurring which is caused by unavoidable inaccuracy of thedistorting, turning, cropping and moving of the currently captured imageto the historical scene.

The result of this image processing and tracking of the features withrespect to the optical flow and the vehicle movement is shown inprinciple in FIG. 2 (and discussed above). The algorithm based onalready established image processing procedures (non-automotive, imageenhancements of photographs and ‘image registration’ and the like), suchas tonal value splitting/-buckling/-limiting, histogram equalization andthe like, as simplified can be seen with reference to FIGS. 3(a) and3(b).

Because the yet to be processed images are captured by a camera on amoving vehicle, it is necessary that the optical flow and the accordinginformation or data of objects (both steady or moving) moving throughthe images, including the vehicle speed, the steering angle of thevehicle and the like, be taken into account. There may be a model of thevehicle's cinematic mathematical equations. Its results may be stored ina look up table. The camera's or cameras parameters as like mountingposition and viewing angle optical properties may be reflected in that(combined) look up table or in another mathematical model or table. Themoving objects/obstacles can thus be distinguished from steady objectsrelative to the movement of the vehicle that is equipped with the camerasystem or vision system of the present invention. Object classificationmay work on further distances by feeding enhanced image data. Furtheralgorithms may process the image data and may indicate hazards or thelike, and/or may actively intervene to avoid collisions and the like.The image enhancing algorithm may find use in processing multiple cameraimages separate or by processing a stitched image which may be arrangedas a vehicle top view image or the like.

The imaging sensor and its photosensor array may comprise any suitablecamera or sensing device, such as, for example, an array of a pluralityof photosensor elements arranged in 640 columns and 480 rows (a 640×480imaging array), with a respective lens focusing images onto respectiveportions of the array. The photosensor array may comprise a plurality ofphotosensor elements arranged in a photosensor array having rows andcolumns. The logic and control circuit of the imaging sensor mayfunction in any known manner, such as in the manner described in U.S.Pat. Nos. 5,550,677; 5,877,897; 6,498,620; 5,670,935; 5,796,094 and/or6,396,397, and/or U.S. provisional applications, Ser. No. 61/696,416,filed Sep. 4, 2012; Ser. No. 61/682,995, filed Aug. 14, 2012; Ser. No.61/682,486, filed Aug. 13, 2012; Ser. No. 61/680,883, filed Aug. 8,2012; Ser. No. 61/678,375, filed Aug. 1, 2012; Ser. No. 61/676,405,filed Jul. 27, 2012; Ser. No. 61/666,146, filed Jun. 29, 2012; Ser. No.61/653,665, filed May 31, 2012; Ser. No. 61/653,664, filed May 31, 2012;Ser. No. 61/648,744, filed May 18, 2012; Ser. No. 61/624,507, filed Apr.16, 2012; Ser. No. 61/616,126, filed Mar. 27, 2012; Ser. No. 61/615,410,filed Mar. 26, 2012; Ser. No. 61/613,651, filed Mar. 21, 2012; Ser. No.61/607,229, filed Mar. 6, 2012; Ser. No. 61/605,409, filed Mar. 1, 2012;Ser. No. 61/602,878, filed Feb. 24, 2012; Ser. No. 61/602,876, filedFeb. 24, 2012; Ser. No. 61/600,205, filed Feb. 17, 2012; Ser. No.61/588,833, filed Jan. 20, 2012; Ser. No. 61/583,381, filed Jan. 5,2012; Ser. No. 61/579,682, filed Dec. 23, 2011; Ser. No. 61/570,017,filed Dec. 13, 2011; Ser. No. 61/568,791, filed Dec. 9, 2011; Ser. No.61/567,446, filed Dec. 6, 2011; Ser. No. 61/559,970, filed Nov. 15,2011; and/or Ser. No. 61/552,167, filed Oct. 27, 2011, and/or PCTApplication No. PCT/CA2012/000378, filed Apr. 25, 2012, and publishedNov. 1, 2012 as International Publication No. WO 2012/145822, and/or PCTApplication No. PCT/US2012/056014, filed Sep. 19, 2012, and publishedMar. 28, 2013 as International Publication No. WO 2013/043661, and/orPCT Application No. PCT/US2012/048800, filed Jul. 30, 2012, andpublished Feb. 7, 2013 as International Publication No. WO 2013/019707,and/or PCT Application No. PCT/US2012/048110, filed Jul. 25, 2012, andpublished Jan. 31, 2013 as International Publication No. WO 2013/016409,and/or U.S. patent application Ser. No. 13/534,657, filed Jun. 27, 2012,and published Jan. 3, 2013 as U.S. Publication No. US-2013-0002873,which are all hereby incorporated herein by reference in theirentireties. The system may communicate with other communication systemsvia any suitable means, such as by utilizing aspects of the systemsdescribed in PCT Application No. PCT/US10/038477, filed Jun. 14, 2010,and/or U.S. patent application Ser. No. 13/202,005, filed Aug. 17, 2011,now U.S. Pat. No. 9,126,525, and/or U.S. provisional applications, Ser.No. 61/650,667, filed May 23, 2012; Ser. No. 61/579,682, filed Dec. 23,2011; Ser. No. 61/565,713, filed Dec. 1, 2011, which are herebyincorporated herein by reference in their entireties.

The imaging device and control and image processor and any associatedillumination source, if applicable, may comprise any suitablecomponents, and may utilize aspects of the cameras and vision systemsdescribed in U.S. Pat. Nos. 5,550,677; 5,877,897; 6,498,620; 5,670,935;5,796,094; 6,396,397; 6,806,452; 6,690,268; 7,005,974; 7,123,168;7,004,606; 6,946,978; 7,038,577; 6,353,392; 6,320,176; 6,313,454 and6,824,281, and/or International Publication No. WO 2010/099416,published Sep. 2, 2010, and/or PCT Application No. PCT/US10/47256, filedAug. 31, 2010, and/or U.S. patent application Ser. No. 12/508,840, filedJul. 24, 2009, and published Jan. 28, 2010 as U.S. Pat. Publication No.US 2010-0020170; and/or PCT Application No. PCT/US2012/048110, filedJul. 25, 2012, and published Jan. 31, 2013 as International PublicationNo. WO 2013/016409, and/or U.S. patent application Ser. No. 13/534,657,filed Jun. 27, 2012, and published Jan. 3, 2013 as U.S. Publication No.US-2013-0002873, which are all hereby incorporated herein by referencein their entireties. The camera or cameras may comprise any suitablecameras or imaging sensors or camera modules, and may utilize aspects ofthe cameras or sensors described in U.S. patent application Ser. No.12/091,359, filed Apr. 24, 2008 and published Oct. 1, 2009 as U.S.Publication No. US-2009-0244361; and/or U.S. patent application Ser. No.13/260,400, filed Sep. 26, 2011, now U.S. Pat. No. 8,542,451, and/orU.S. Pat. Nos. 7,965,336 and/or 7,480,149, which are hereby incorporatedherein by reference in their entireties. The imaging array sensor maycomprise any suitable sensor, and may utilize various imaging sensors orimaging array sensors or cameras or the like, such as a CMOS imagingarray sensor, a CCD sensor or other sensors or the like, such as thetypes described in U.S. Pat. Nos. 5,550,677; 5,670,935; 5,760,962;5,715,093; 5,877,897; 6,922,292; 6,757,109; 6,717,610; 6,590,719;6,201,642; 6,498,620; 5,796,094; 6,097,023; 6,320,176; 6,559,435;6,831,261; 6,806,452; 6,396,397; 6,822,563; 6,946,978; 7,339,149;7,038,577; 7,004,606; 7,720,580 and/or 7,965,336, and/or PCT ApplicationNo. PCT/US2008/076022, filed Sep. 11, 2008 and published Mar. 19, 2009as International Publication No. WO 2009/036176, and/or PCT ApplicationNo. PCT/US2008/078700, filed Oct. 3, 2008 and published Apr. 9, 2009 asInternational Publication No. WO 2009/046268, which are all herebyincorporated herein by reference in their entireties.

The camera module and circuit chip or board and imaging sensor may beimplemented and operated in connection with various vehicularvision-based systems, and/or may be operable utilizing the principles ofsuch other vehicular systems, such as a vehicle headlamp control system,such as the type disclosed in U.S. Pat. Nos. 5,796,094; 6,097,023;6,320,176; 6,559,435; 6,831,261; 7,004,606; 7,339,149 and/or 7,526,103,which are all hereby incorporated herein by reference in theirentireties, a rain sensor, such as the types disclosed in commonlyassigned U.S. Pat. Nos. 6,353,392; 6,313,454; 6,320,176 and/or7,480,149, which are hereby incorporated herein by reference in theirentireties, a vehicle vision system, such as a forwardly, sidewardly orrearwardly directed vehicle vision system utilizing principles disclosedin U.S. Pat. Nos. 5,550,677; 5,670,935; 5,760,962; 5,877,897; 5,949,331;6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202;6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452;6,822,563; 6,891,563; 6,946,978 and/or 7,859,565, which are all herebyincorporated herein by reference in their entireties, a trailer hitchingaid or tow check system, such as the type disclosed in U.S. Pat. No.7,005,974, which is hereby incorporated herein by reference in itsentirety, a reverse or sideward imaging system, such as for a lanechange assistance system or lane departure warning system or for a blindspot or object detection system, such as imaging or detection systems ofthe types disclosed in U.S. Pat. Nos. 7,881,496; 7,720,580; 7,038,577;5,929,786 and/or 5,786,772, and/or U.S. provisional applications, Ser.No. 60/628,709, filed Nov. 17, 2004; Ser. No. 60/614,644, filed Sep. 30,2004; Ser. No. 60/618,686, filed Oct. 14, 2004; Ser. No. 60/638,687,filed Dec. 23, 2004, which are hereby incorporated herein by referencein their entireties, a video device for internal cabin surveillanceand/or video telephone function, such as disclosed in U.S. Pat. Nos.5,760,962; 5,877,897; 6,690,268 and/or 7,370,983, and/or U.S. patentapplication Ser. No. 10/538,724, filed Jun. 13, 2005 and published Mar.9, 2006 as U.S. Publication No. US-2006-0050018, which are herebyincorporated herein by reference in their entireties, a traffic signrecognition system, a system for determining a distance to a leading ortrailing vehicle or object, such as a system utilizing the principlesdisclosed in U.S. Pat. Nos. 6,396,397 and/or 7,123,168, which are herebyincorporated herein by reference in their entireties, and/or the like.

Optionally, the circuit board or chip may include circuitry for theimaging array sensor and or other electronic accessories or features,such as by utilizing compass-on-a-chip or EC driver-on-a-chip technologyand aspects such as described in U.S. Pat. Nos. 7,255,451 and/or7,480,149; and/or U.S. patent application Ser. No. 11/226,628, filedSep. 14, 2005 and published Mar. 23, 2006 as U.S. Publication No.US-2006-0061008, and/or U.S. patent application Ser. No. 12/578,732,filed Oct. 14, 2009, now U.S. Pat. No. 9,487,144, which are herebyincorporated herein by reference in their entireties.

Optionally, the vision system may include a display for displayingimages captured by one or more of the imaging sensors for viewing by thedriver of the vehicle while the driver is normally operating thevehicle. Optionally, for example, the vision system may include a videodisplay device disposed at or in the interior rearview mirror assemblyof the vehicle, such as by utilizing aspects of the video mirror displaysystems described in U.S. Pat. No. 6,690,268 and/or U.S. patentapplication Ser. No. 13/333,337, filed Dec. 21, 2011, now U.S. Pat. No.9,264,672, which are hereby incorporated herein by reference in theirentireties. The video mirror display may comprise any suitable devicesand systems and optionally may utilize aspects of the compass displaysystems described in U.S. Pat. Nos. 7,370,983; 7,329,013; 7,308,341;7,289,037; 7,249,860; 7,004,593; 4,546,551; 5,699,044; 4,953,305;5,576,687; 5,632,092; 5,677,851; 5,708,410; 5,737,226; 5,802,727;5,878,370; 6,087,953; 6,173,508; 6,222,460; 6,513,252 and/or 6,642,851,and/or European patent application, published Oct. 11, 2000 underPublication No. EP 0 1043566, and/or U.S. patent application Ser. No.11/226,628, filed Sep. 14, 2005 and published Mar. 23, 2006 as U.S.Publication No. US-2006-0061008, which are all hereby incorporatedherein by reference in their entireties. Optionally, the video mirrordisplay screen or device may be operable to display images captured by arearward viewing camera of the vehicle during a reversing maneuver ofthe vehicle (such as responsive to the vehicle gear actuator beingplaced in a reverse gear position or the like) to assist the driver inbacking up the vehicle, and optionally may be operable to display thecompass heading or directional heading character or icon when thevehicle is not undertaking a reversing maneuver, such as when thevehicle is being driven in a forward direction along a road (such as byutilizing aspects of the display system described in PCT Application No.PCT/US2011/056295, filed Oct. 14, 2011 and published Apr. 19, 2012 asInternational Publication No. WO 2012/051500, which is herebyincorporated herein by reference in its entirety).

Optionally, the vision system (utilizing the forward facing camera and arearward facing camera and other cameras disposed at the vehicle withexterior fields of view) may be part of or may provide a display of atop-down view or birds-eye view system of the vehicle or a surround viewat the vehicle, such as by utilizing aspects of the vision systemsdescribed in PCT Application No. PCT/US10/25545, filed Feb. 26, 2010 andpublished on Sep. 2, 2010 as International Publication No. WO2010/099416, and/or PCT Application No. PCT/US10/47256, filed Aug. 31,2010 and published Mar. 10, 2011 as International Publication No. WO2011/028686, and/or PCT Application No. PCT/US11/62755, filed Dec. 1,2011 and published Jun. 7, 2012 as International Publication No. WO2012-075250, and/or PCT Application No. PCT/US2012/048993, filed Jul.31, 2012, and published Feb. 7, 2013 as International Publication No. WO2013/019795, and/or PCT Application No. PCT/CA2012/000378, filed Apr.25, 2012, and published Nov. 1, 2012 as International Publication No. WO2012/145822, and/or U.S. patent application Ser. No. 13/333,337, filedDec. 21, 2011, now U.S. Pat. No. 9,264,672, and/or U.S. provisionalapplications, Ser. No. 61/615,410, filed Mar. 26, 2012; Ser. No.61/588,833, filed Jan. 20, 2012; Ser. No. 61/570,017, filed Dec. 13,2011; Ser. No. 61/568,791, filed Dec. 9, 2011; Ser. No. 61/559,970,filed Nov. 15, 2011; Ser. No. 61/540,256, filed Sep. 28, 2011, which arehereby incorporated herein by reference in their entireties.

Optionally, the video mirror display may be disposed rearward of andbehind the reflective element assembly and may comprise a display suchas the types disclosed in U.S. Pat. Nos. 5,530,240; 6,329,925;7,855,755; 7,626,749; 7,581,859; 7,338,177; 7,274,501; 7,255,451;7,195,381; 7,184,190; 5,668,663; 5,724,187 and/or 6,690,268, and/or inU.S. patent applications, Ser. No. 11/226,628, filed Sep. 14, 2005 andpublished Mar. 23, 2006 as U.S. Publication No. US-2006-0061008; and/orSer. No. 10/538,724, filed Jun. 13, 2005 and published Mar. 9, 2006 asU.S. Publication No. US-2006-0050018, which are all hereby incorporatedherein by reference in their entireties. The display is viewable throughthe reflective element when the display is activated to displayinformation. The display element may be any type of display element,such as a vacuum fluorescent (VF) display element, a light emittingdiode (LED) display element, such as an organic light emitting diode(OLED) or an inorganic light emitting diode, an electroluminescent (EL)display element, a liquid crystal display (LCD) element, a video screendisplay element or backlit thin film transistor (TFT) display element orthe like, and may be operable to display various information (asdiscrete characters, icons or the like, or in a multi-pixel manner) tothe driver of the vehicle, such as passenger side inflatable restraint(PSIR) information, tire pressure status, and/or the like. The mirrorassembly and/or display may utilize aspects described in U.S. Pat. Nos.7,184,190; 7,255,451; 7,446,924 and/or 7,338,177, which are all herebyincorporated herein by reference in their entireties. The thicknessesand materials of the coatings on the substrates of the reflectiveelement may be selected to provide a desired color or tint to the mirrorreflective element, such as a blue colored reflector, such as is knownin the art and such as described in U.S. Pat. Nos. 5,910,854; 6,420,036and/or 7,274,501, which are hereby incorporated herein by reference intheir entireties.

Optionally, the display or displays and any associated user inputs maybe associated with various accessories or systems, such as, for example,a tire pressure monitoring system or a passenger air bag status or agarage door opening system or a telematics system or any other accessoryor system of the mirror assembly or of the vehicle or of an accessorymodule or console of the vehicle, such as an accessory module or consoleof the types described in U.S. Pat. Nos. 7,289,037; 6,877,888;6,824,281; 6,690,268; 6,672,744; 6,386,742 and 6,124,886, and/or U.S.patent application Ser. No. 10/538,724, filed Jun. 13, 2005 andpublished Mar. 9, 2006 as U.S. Publication No. US-2006-0050018, whichare hereby incorporated herein by reference in their entireties.

The display or displays may comprise a video display and may utilizeaspects of the video display devices or modules described in U.S. Pat.Nos. 6,690,268; 7,184,190; 7,274,501; 7,370,983; 7,446,650 and/or7,855,755, and/or U.S. patent application Ser. No. 10/538,724, filedJun. 13, 2005 and published Mar. 9, 2006 as U.S. Publication No.US-2006-0050018, which are all hereby incorporated herein by referencein their entireties. The video display may be operable to display imagescaptured by one or more imaging sensors or cameras at the vehicle.

Changes and modifications to the specifically described embodiments maybe carried out without departing from the principles of the presentinvention, which is intended to be limited only by the scope of theappended claims as interpreted according to the principles of patentlaw.

The invention claimed is:
 1. A method of image enhancement for a vehiclevision system, said method comprising: (a) providing a camera at avehicle so that the camera has an exterior field of view; (b) providinga processor operable to process image data; (c) capturing multipleframes of image data with the camera; (d) enhancing contrast in imagedata by tone mapping; (e) tracking, as the vehicle moves, contrastthresholds within the captured frames of image data with respect toimage flow caused by the vehicle's movement; (f) passing image data of afirst frame of captured image data through two individual image transferfunctions to generate a first transferred frame of image data; (g)blending the first transferred frame with a second frame of image data;(h) detecting, at least via processing of image data by the processor,presence of an object in the field of view of the camera; and (i)generating an output responsive to detection of the object present inthe field of view of the camera.
 2. The method of claim 1, comprisingexecuting a brightness transfer function to enhance contrast of imagedata.
 3. The method of claim 1, comprising executing tone mapping ofmultiple frames of image data to enhance detection of the object presentin the field of view of the camera.
 4. The method of claim 1, comprisingclassifying the detected object present in the field of view of thecamera.
 5. The method of claim 4, comprising generating an outputresponsive to classification of the detected object.
 6. The method ofclaim 1, comprising determining a low visibility driving condition and,responsive to determination of the low visibility driving condition,increasing contrast of features in captured image data by brighteningbrighter areas of image data and dampening darker areas of image data.7. The method of claim 6, comprising increasing contrast of featuresover multiple successive frames of captured image data.
 8. The method ofclaim 7, comprising tracking, via processing by the processor ofmultiple successive frames of captured image data during the determinedlow visibility driving condition, image flow caused by movement of thevehicle to enhance detection and identification of objects present inthe field of view of the camera.
 9. The method of claim 1, wherein atleast one previously captured frame of image data is retrieved frommemory.
 10. The method of claim 1, comprising determining a lowvisibility driving condition via processing by the processor of imagedata.
 11. The method of claim 1, comprising determining that fog ispresent in the field of view of the camera via processing by theprocessor of image data.
 12. The method of claim 1, comprising providingblended frames of image data to a video display screen that is disposedin the vehicle at a location viewable by a driver of the vehicle whenoperating the vehicle.
 13. The method of claim 1, wherein capturingmultiple frames of image data with the camera comprises capturingmultiple frames of image data with the camera at a frame rate of atleast 30 frames per second.
 14. The method of claim 1, wherein blendingthe first transferred frame with the second frame of image datagenerates a blended image frame of image data that is up to 20 percentderived from the first frame of image data.
 15. The method of claim 1,comprising executing a brightness transfer function on at least onepreviously captured frame of image data.
 16. The method of claim 1,comprising providing the generated output to a driver assistance systemof the vehicle.
 17. The method of claim 16, wherein the driverassistance system of the vehicle comprises a system selected from thegroup consisting of (i) a lane change assist system of the vehicle, (ii)a lane departure warning system of the vehicle, (iii) a blind spotdetection system of the vehicle, (iv) an adaptive cruise control systemof the vehicle, (v) a collision avoidance system of the vehicle, (vi) atraffic sign recognition system of the vehicle and (vii) a vehicleheadlamp control system of the vehicle.
 18. The method of claim 1,comprising tracking the detected object over successive frames of imagedata to determine if the detected object is an object of interest in thefield of view of the camera.
 19. The method of claim 1, whereinprocessing by the processor of image data by the processor is responsiveat least in part to steering of the vehicle.
 20. The method of claim 1,comprising distinguishing, via processing by the processor of imagedata, moving objects from non-moving objects.
 21. The method of claim20, wherein distinguishing moving objects comprises distinguishingmoving objects responsive at least in part to at least one of (i) speedof the vehicle and (ii) steering of the vehicle.
 22. The method of claim1, wherein providing the camera at the vehicle comprises disposing thecamera at a rear portion of the vehicle with an exterior field of viewat least rearward of the vehicle, and wherein said method comprisesproviding a plurality of cameras at the vehicle so as to have respectiveexterior fields of view, and wherein the plurality of cameras comprisesthe camera at the rear portion of the vehicle.
 23. The method of claim22, comprising providing a display for displaying images derived, atleast in part, from image data captured by the camera at the rearportion of the vehicle and derived, at least in part, from image datacaptured by other cameras of the plurality of cameras.
 24. The method ofclaim 1, wherein providing the camera at the vehicle comprises disposingthe camera at a rear portion of the vehicle with an exterior field ofview at least rearward of the vehicle, and wherein said method comprisesproviding a display for displaying images derived, at least in part,from image data captured by the camera during a reversing maneuver ofthe vehicle.
 25. A method of image enhancement for a vehicle visionsystem, said method comprising: (a) disposing a camera at a frontportion of a vehicle with an exterior field of view at least forward ofthe vehicle; (b) providing a processor operable to process image data;(c) capturing multiple frames of image data with the camera; (d)executing a brightness transfer function on at least one frame of imagedata; (e) tracking, as the vehicle moves, image flow caused by thevehicle's movement; (f) detecting, at least via processing of image databy the processor, presence of an object in the field of view of thecamera; (g) tracking the detected object over successive frames of imagedata to determine if the detected object is an object of interest in thefield of view of the camera; (h) generating an output responsive todetection of the object present in the field of view of the camera; (i)providing the generated output to a driver assistance system of thevehicle, wherein the driver assistance system of the vehicle comprises asystem selected from the group consisting of (i) a lane change assistsystem of the vehicle, (ii) a lane departure warning system of thevehicle, (iii) a blind spot detection system of the vehicle, (iv) anadaptive cruise control system of the vehicle, (v) a collision avoidancesystem of the vehicle, (vi) a traffic sign recognition system of thevehicle and (vii) a vehicle headlamp control system of the vehicle; (j)determining a low visibility driving condition via processing of imagedata by the processor; (k) responsive to determination of the lowvisibility driving condition, increasing contrast of features incaptured image data by brightening brighter areas of image data anddampening darker areas of image data; and (l) distinguishing, at leastvia processing of image data by the processor, moving objects fromnon-moving objects, and wherein distinguishing moving objects comprisesdistinguishing moving objects responsive at least in part to at leastone of (i) speed of the vehicle and (ii) steering of the vehicle.
 26. Amethod of image enhancement for a vehicle vision system, said methodcomprising: (a) disposing a plurality of cameras at the vehicle so as tohave respective exterior fields of view, wherein the plurality ofcameras comprises a rear camera at a rear portion of a vehicle with anexterior field of view at least rearward of the vehicle; (b) providing aprocessor operable to process image data; (c) capturing multiple framesof image data with the rear camera; (d) executing tone mapping ofmultiple frames of image data to enhance contrast; (e) detectingpresence of an object in the field of view of the rear camera; (f)generating an output responsive to detection of the object present inthe field of view of the rear camera; (g) providing a display fordisplaying images derived, at least in part, from image data captured bythe rear camera at the rear portion of the vehicle and derived, at leastin part, from image data captured by other cameras of the plurality ofcameras; (h) determining a low visibility driving condition viaprocessing by the processor of image data captured by at least one ofthe plurality of cameras; (i) responsive to determination of the lowvisibility driving condition, increasing contrast of features incaptured image data by brightening brighter areas of image data anddampening darker areas of image data; and (j) distinguishing, at leastvia processing of image data by the processor, moving objects fromnon-moving objects, and wherein distinguishing moving objects comprisesdistinguishing moving objects responsive at least in part to at leastone of (i) speed of the vehicle and (ii) steering of the vehicle.